Think Believe 2 (free response)

kable In this notebook we analyze the second “think/believe” task, in which participants completed a series of fill-in-the-blanks with free responses.

NAs introduced by coercion

Note: There are some subjective calls here, and translation issues. For now, I’ve used a combination of automatic “lemmatization” and hand-coding to do my best to code whether responses should count as “believe” (e.g., including obvious cases like “believes” and “believed”; including Bislama “belif” and “bilif” and other spellings).

As of 2020-01-22, I have two versions of these variables – one that only includes responses that are strictly “believe” and one that also includes cases that involve the word “believe” (e.g., “firmly believes,” “does not believe”) – and likewise for think. The looser interpretations are called believeX (or “belief*”) and thinkX (or “think*”).

Overview

From the preregistration (link):

“Our overarching hypothesis for the present study is that […] other languages will have an epistemic verb that is more likely to be used for religious attitude reports (similar to English “believe”) and a different epistemic verb that is more likely to be used for matter-of-fact attitude reports (similar to English “think”).

For this study, we are examining five languages in five regions of interest: (i) Mandarin in China; (ii) Thai in Thailand; (iii) Bislama (an English-based creole language) on the Melanesian Island of Vanuatu; (iv) Fante in Ghana; and (v) American English in the Bay Area, California.

We thus have five more specific sub-hypotheses. For each of the first four languages / regions of interest, we hypothesize that a set of words or phrases exists whose usage parallels the difference between usage of “think” and “believe” in American English, with one word or phrase (the “think” analogue) being used for more matter-of-fact attitude reports and the other (the “believe” analogue) being more likely to be used for religious attitude reports. That gives us our first four sub-hypotheses: that Mandarin, Thai, Bislama and Fante speakers will each use two different words in a manner parallel to the use of “think” and “believe” in an American English setting as identified by Heiphetz, Landers, and Van Leeuwen. Our fifth sub-hypothesis is that the Bay Area portion of the study will replicate the results of the earlier study of Heiphetz, Landers, and Van Leeuwen."

KW EXECUTIVE SUMMARY (2020-01-20): We replicated the original finding in the US (and the findings of Think Believe 1): participants were more likely to write in “believe” for religious than fact questions. We found the same pattern in all five countries/langauges included in this study.

As in Think Believe 1, the pattern was somewhat weaker in Ghana/Fante than in other countries/languages. In Think Believe 1, the pattern was stronger in Thailand/Thai (and no stronger or weaker in China/Mandarin or Vanuatu/Bislama); in contrast, in this study it was stronger in China/Mandarin, weaker in Vanuatu/Bislama, and no stronger or weaker Thailand/Thai. I suspect these patterns are largely accounted for by the fact that so few participants in Ghana and especially Vanuatu spontaneously used the word “believe” in their free responses.

Samples

Before we begin, it’s important to note that we had unequal sample sizes by country:

However, 39 participants completed this task after completing other surveys, and an additional 31 failed the attention check. In the following analyses I will exclude these participants, leaving us with the following samples:

Plots

We’ll begin by plotting responses of “think(s)/thought” (red) vs. “believe(s)/believed” (turquoise) vs. other responses (gray) to get an overall sense of any patterns in the data.

By superordinate category

By category

By question

Analysis: KW without looking at preregistration

These analyses directly parallel the way I analyzed the Think Believe 1 data before looking at the preregistration. Again, I think these analyses are valuable because they’re a little more efficient than the preregistered analyses – no need for follow-up tests – and they directly test the question of whether the effect of interest varies across countries/langauges.

As of 2020-01-22, I’m now using the more lenient “believe*” variable in these analyses.

Technical note: Unless specified otherwise, all of these analyses use “effect coding” for categorical variables (e.g., country, category of question) – meaning that each country/langauge is compared to the “grand mean” collapsing across all countries/languages. Because of degrees of freedom issues, each analysis only compares 4 of the 5 countries to the grand mean – by default, I’ve left out the comparison of the US/English to the grand mean, but stats for that comparison could easily be calculated (if we left out another country/language instead). This is just to say that you won’t see statements like “The effect was exaggerated in the US relative to other countries,” although they might be true.

KW Analysis #1

First, I used a mixed effects logistic regression predicting how likely a participant was to write “believe” based on the superordinate category of the question (“religious” questions or “fact” questions), the country they were in/language they were using (US/English, Ghana/Fante, Thailand/Thai, China/Mandarin, or Vanuatu/Bislama), and an interaction between them, with a maximal random effects structure (random interpcepts and slopes by subject, and random intercepts by question). This analysis gives me a sense of (1) Whether participants were more likely to write “believe” for religious questions than fact questions, and whether this tendency varied by country/language, controlling for the fact that the overall rates of circling “believe” might vary by country/language (and accounting for individual differences and differences across individual questions).

Note that this analysis treats responses of “think” as the same as any other non-“believe” response – I’m just trying to predict how likey the participant was to write in “believe.”

r2.1 <- lmer(believeX ~ super_cat * country 
             + (1 + super_cat | thb2_subj) + (1 | question), 
             # + (1 + super_cat || thb2_subj) + (1 | question), 
             # + (1 + super_cat | thb2_subj), 
             # + (1 + super_cat || thb2_subj), 
             # + (1 | thb2_subj) + (1 | question), 
             data = d2_long)

funs() is soft deprecated as of dplyr 0.8.0
Please use a list of either functions or lambdas: 

  # Simple named list: 
  list(mean = mean, median = median)

  # Auto named with `tibble::lst()`: 
  tibble::lst(mean, median)

  # Using lambdas
  list(~ mean(., trim = .2), ~ median(., na.rm = TRUE))
This warning is displayed once per session.

Parameter	β	β'	β''	Std. Err.	df	t	p
Intercept	0.19	-	-	0.01	55.49	13.80	<0.001	***
Category (religious)	0.13	0.35	0.35	0.01	40.79	10.58	<0.001	***
Country (Gh.)	-0.09	-0.14	-0.14	0.02	383.00	-4.57	<0.001	***
Country (Th.)	0.01	0.01	0.01	0.02	383.00	0.55	0.584
Country (Ch.)	0.03	0.05	0.05	0.02	383.00	1.80	0.073
Country (Vt.)	-0.03	-0.06	-0.06	0.02	383.00	-2.03	0.043	*
Category (religious) × Country (Gh.)	-0.07	-0.11	-0.11	0.02	383.01	-4.53	<0.001	***
Category (religious) × Country (Th.)	0.01	0.01	0.01	0.01	383.01	0.61	0.541
Category (religious) × Country (Ch.)	0.04	0.07	0.07	0.01	383.01	3.24	0.001	**
Category (religious) × Country (Vt.)	-0.04	-0.07	-0.07	0.01	383.01	-3.05	0.002	**

The effects of primary interest are in bold:

• Category (religious): Collapsing across countries/languages, participants were indeed more likely to say “believe” for “religious” questions, echoing the forced choice results of Think Believe 1.
• Country (Gh.): Participants in Ghana were generally less likely than other participants to say “believe,” collapsing across question categories. (This is in contrast to Think Believe 1, in which they were more likely to circle “believe.”)
• Country (Th.): Participants in Thailand were no more or less likely than other participants to say “believe,” collapsing across question categories. (This is in contrast to Think Believe 1, in which they were less likely to circle “believe.”)
• Country (Ch.): Participants in China were no more or less likely than other participants to say “believe,” caollapsing across question categories. (They did not differ from the grand mean in Think Believe 1.)
• Country (Vt.): Participants in Vanuatu were no more or less likely than other participants to say “believe,” collapsing across question categories. (They did not differ from the grand mean in Think Believe 1.)
• Category (religious) x Country (Gh.): The difference in rates of “believe” responses between question categories was smaller in Ghana than in other countries, echoing the forced choice results of Think Believe 1.
• Category (religious) x Country (Th.): The difference in rates of “believe” responses between question categories was no smaller or larger in Thailand than in other countries. (In Think Believe 1, the difference was exaggerated in Thailand.)
• Category (religious) x Country (Ch.): The difference in rates of “believe” responses between question categories was larger in China than in other countries. (In Think Believe 1, this difference did not differ from the difference in other countries.)
• Category (religious) x Country (Vt.): The difference in rates of “believe” responses between question categories was smaller in Vanuatu than in other countries. (In Think Believe 1, this difference did not differ from the difference in other countries.)

Take-away: The predicted effect is evident in this dataset, as it was in Think Believe 1. It appears to be exaggerated in China and diminished in Ghana and Vanuatu, a pattern which differs from Think Believe 1.

KW Analyses #1a-1e (by country)

Next, I did this same analysis within each country/langauge alone (using the most maximal random effect structure that converged across all countries/languages).

# note: using most maximal common random effects structure
r2.1_us <- lmer(believeX ~ super_cat + 
                  # (1 + super_cat | thb2_subj) + (1 | question),
                  # (1 + super_cat || thb2_subj) + (1 | question),
                  # (1 | thb2_subj) + (1 | question),
                  (1 + super_cat | thb2_subj),
                  # (1 + super_cat || thb2_subj),
                  # (1 | thb2_subj),
                data = d2_long %>% filter(country == "US"))

r2.1_gh <- lmer(believeX ~ super_cat + 
                  # (1 + super_cat | thb2_subj) + (1 | question),
                  # (1 + super_cat || thb2_subj) + (1 | question),
                  # (1 | thb2_subj) + (1 | question),
                  (1 + super_cat | thb2_subj),
                  # (1 + super_cat || thb2_subj), 
                  # (1 | thb2_subj),
                data = d2_long %>% filter(country == "Ghana"))

r2.1_th <- lmer(believeX ~ super_cat + 
                  # (1 + super_cat | thb2_subj) + (1 | question), # failed to converge
                  # (1 + super_cat || thb2_subj) + (1 | question), # failed to converge
                  (1 | thb2_subj) + (1 | question),
                  # (1 + super_cat | thb2_subj),
                  # (1 + super_cat || thb2_subj), 
                  # (1 | thb2_subj),
                data = d2_long %>% filter(country == "Thailand"))

r2.1_ch <- lmer(believeX ~ super_cat + 
                  # (1 + super_cat | thb2_subj) + (1 | question),
                  # (1 + super_cat || thb2_subj) + (1 | question),
                  # (1 | thb2_subj) + (1 | question), # failed to converge
                  (1 + super_cat | thb2_subj),
                  # (1 + super_cat || thb2_subj), 
                  # (1 | thb2_subj),
                data = d2_long %>% filter(country == "China"))

r2.1_vt <- lmer(believeX ~ super_cat + 
                  # (1 + super_cat | thb2_subj) + (1 | question),
                  # (1 + super_cat || thb2_subj) + (1 | question),
                  # (1 | thb2_subj) + (1 | question),
                  (1 + super_cat | thb2_subj),
                  # (1 + super_cat || thb2_subj), 
                  # (1 | thb2_subj),
                data = d2_long %>% filter(country == "Vanuatu"))

Country	Parameter	β	Std. Err.	df	t	p
US	Intercept	0.28	0.02	69.98	13.90	<0.001	***
	Category (religious)	0.20	0.02	70.00	11.48	<0.001	***
Ghana	Intercept	0.09	0.02	44.99	4.20	<0.001	***
	Category (religious)	0.06	0.01	45.00	4.06	<0.001	***
Thailand	Intercept	0.19	0.02	78.96	9.90	<0.001	***
	Category (religious)	0.14	0.01	23.00	10.83	<0.001	***
China	Intercept	0.21	0.02	99.86	14.03	<0.001	***
	Category (religious)	0.17	0.01	100.73	12.32	<0.001	***
Vanuatu	Intercept	0.15	0.02	71.98	8.16	<0.001	***
	Category (religious)	0.09	0.01	72.00	6.83	<0.001	***

The effects of primary interest are in bold, and the take-away is clear: In every country/language, participants were more likely to say “believe” in “religious” questions than in “fact” questions.

KW Analysis #2

In this analysis, I treated country/language as a random rather than fixed effect (with participants nested within countries).

r2.2 <- lmer(believeX ~ super_cat 
             # + (1 + super_cat | country/thb2_subj) + (1 | question), # failed to converge
             # + (1 + super_cat || country/thb2_subj) + (1 | question), # failed to converge
             # + (1 + super_cat | country/thb2_subj), # failed to converge
             # + (1 + super_cat || country/thb2_subj), # failed to converge
             # + (1 | country/thb2_subj) + (1 | question), # failed to converge
             + (1 | country/thb2_subj), 
             data = d2_long)

Parameter	β	Std. Err.	df	t	p
Intercept	0.19	0.03	3.83	7.34	0.002	**
Category (religious)	0.14	0.00	9311.02	42.15	<0.001	***

The effect still holds.

KW Analysis #3

Finally, I ran a version of this first model looking at 5 categories of questions (rather than 2 superordinate categories): Christian religious, Buddhist religious, well-known fact, esoteric fact, and personal fact. I compared these categories using planned orthogonal contrasts.

r2.3 <- lmer(believeX ~ category * country 
             + (1 + category | thb2_subj) + (1 | question), 
             # + (1 + category || thb2_subj) + (1 | question),
             # + (1 + category | thb2_subj), 
             # + (1 + category || thb2_subj), 
             # + (1 | thb2_subj) + (1 | question),
             data = d2_long)

Parameter	β	Std. Err.	df	t	p
Intercept
Intercept	0.16	0.01	53.95	14.60	<0.001	***
Category comparisons
Category (Religious vs. fact)	0.05	0.00	44.39	12.41	<0.001	***
Category (Christian vs. Buddhist religious)	0.05	0.01	25.53	3.18	0.004	**
Category (well-known & esoteric vs. personal fact)	0.01	0.01	20.68	1.78	0.090
Category (well-known vs. esoteric fact)	0.01	0.01	20.50	0.55	0.588
Country comparisons
Country (Gh.)	-0.08	0.02	382.94	-4.45	<0.001	***
Country (Th.)	0.01	0.01	382.94	0.52	0.603
Country (Ch.)	0.02	0.01	382.94	1.48	0.140
Country (Vt.)	-0.03	0.01	382.94	-1.79	0.074
Interactions: Ghana
Category (Religious vs. fact) × Country (Gh.)	-0.03	0.01	384.54	-4.62	<0.001	***
Category (Christian vs. Buddhist religious) × Country (Gh.)	-0.02	0.02	382.99	-1.21	0.227
Category (well-known & esoteric vs. personal fact) × Country (Gh.)	0.00	0.01	3216.39	-0.44	0.657
Category (well-known vs. esoteric fact) × Country (Gh.)	0.00	0.01	7596.51	-0.08	0.938
Interactions: Thailand
Category (Religious vs. fact) × Country (Th.)	0.00	0.00	384.54	0.62	0.533
Category (Christian vs. Buddhist religious) × Country (Th.)	-0.03	0.01	382.99	-2.61	0.009	**
Category (well-known & esoteric vs. personal fact) × Country (Th.)	0.00	0.00	3216.39	-0.93	0.353
Category (well-known vs. esoteric fact) × Country (Th.)	0.00	0.01	7596.51	0.21	0.834
Interactions: China
Category (Religious vs. fact) × Country (Ch.)	0.02	0.00	384.54	3.31	0.001	**
Category (Christian vs. Buddhist religious) × Country (Ch.)	0.02	0.01	382.99	1.94	0.053
Category (well-known & esoteric vs. personal fact) × Country (Ch.)	0.00	0.00	3216.39	-0.17	0.867
Category (well-known vs. esoteric fact) × Country (Ch.)	0.01	0.01	7596.51	0.66	0.508
Interactions: Vanuatu
Category (Religious vs. fact) × Country (Vt.)	-0.02	0.01	384.54	-3.11	0.002	**
Category (Christian vs. Buddhist religious) × Country (Vt.)	0.03	0.01	382.99	2.29	0.023	*
Category (well-known & esoteric vs. personal fact) × Country (Vt.)	0.00	0.01	3216.39	0.80	0.425
Category (well-known vs. esoteric fact) × Country (Vt.)	0.00	0.01	7596.52	-0.20	0.839

The first orthogonal contrast compared the two “religious” categories to the three “fact” categories (“Category (Religoius vs. fact)”). This parallels the previous analyses, and the results are similar: Overall, participants were more likely to write “believe” for religious questions than fact questions, and this tendency was diminished in Ghana and Vanuatu, and exaggerated in China.

The second orthogonal contrast compared Christian to Buddhist “religious” questions. Overall, participants were more likely to write “believe” for Christian questions, and this tendency was exaggerated in Vanuatu and diminished in Thailand (partially echoing Think Believe 1).

The third orthogonal contrast compared well-known and esoteric facts, on the one hand, to personal facts, on the other. Overall, there was no reliable difference in rates of “believe” between these groups of questions (in contrast to Think Believe 1, in which participants were more likely to circle “believe” for well-known and esoteric facts). This difference did not vary by country.

The fourth orthogonal contrast compared well-known to esoteric facts. Overall, there was no reliable difference in rates of “believe” between these groups of questions (in contrast to Think Believe 1, in which participants were more likely to circle “believe” for well-known facts). This difference did not vary by country.

Note that these findings statistically control for differences across samples in the overall rate of writing “believe” (which was generally lower in Ghana).

Analysis: Based on preregistration

From preregistration:

“Survey 1: We will conduct a 5 (Site: China vs. Thailand vs. Vanuatu vs. Ghana vs. United States) x 2 (Statement Type: religion vs. fact) mixed ANOVA with repeated measures on the second factor and the proportion of trials on which participants completed sentences using a form the word “believe” (or its respective translation) as the dependent measure. To look for finer-grained differences between different religious and factual statements, we will also conduct a 5 (Site: China vs. Thailand vs. Vanuatu vs. Ghana vs. United States) x 5 (Statement Type: Buddhist religious statements vs. Christian religious statements vs. life facts vs. well-known facts vs. esoteric facts) mixed ANOVA with repeated measures on the second factor and the proportion of trials on which participants completed sentences using a form of the word “believe” (or its respective translation) as the dependent measure. In all cases where omnibus ANOVAs are significant, we will conduct pairwise analyses comparing each statement type with each other statement type and each site with each other site."

d2_anova <- d2_long %>%
  distinct(thb2_subj, country, super_cat, question, believeX) %>%
  group_by(thb2_subj, country, super_cat) %>%
  summarise(prop_believeX = mean(believeX)) %>%
  ungroup() %>%
  mutate(thb2_subj = factor(thb2_subj))

contrasts(d2_anova$country) <- contrast_country
contrasts(d2_anova$super_cat) <- contrast_super_cat

Prereg Analysis #1

Here is the first preregistered analyis: a 5 (country) x 2 (question category) mixed ANOVA with repeated measures on the second factor and the proportion of trials on which participants wrote “berlieve” as the DV.

r2.4 <- d2_anova %>%
  anova_test(dv = prop_believeX, 
             wid = thb2_subj, 
             between = country, 
             within = super_cat)

get_anova_table(r2.4)

ANOVA Table (type III tests)

             Effect DFn DFd       F        p p<.05   ges
1           country   4 383  10.868 2.35e-08     * 0.065
2         super_cat   1 383 359.606 5.00e-57     * 0.268
3 country:super_cat   4 383  12.586 1.25e-09     * 0.049

This analysis aligns with the regressions above and with Think Believe 1, suggesting that participants’ tendency to write “believe” varied by country/language (country) and by question category (super_cat), and the difference between question category varied across countries/languages (i.e., there was an interaction: country:super_cat).

The preregistration indicated that we’d conduct pairwise follow-up analyses comparing the two question categories and comparing pairs of countires/languages – but, again, I don’t really think we’re interested in comparing pairs of countries/languages, so I’m going to skip that for now. Instead, I’ll compare the two questions categories within each country/language (to explore the significant interaction), as I did for Think Believe 1.

Here we go:

Comparing question categories

r2.5a <- t.test(prop_believeX ~ super_cat, paired = T, d2_anova); r2.5a

    Paired t-test

data:  prop_believeX by super_cat
t = 19.793, df = 387, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 0.2509326 0.3062839
sample estimates:
mean of the differences 
              0.2786082 

Collapsing across countries/languages, participants wrote significantly more “believe” responses for questions in the religious category (28%) than they did for questions in the fact category (NA%).

Comparing question categories within countries/languages

# US
r2.5b_us <- t.test(prop_believeX ~ super_cat, paired = T,
                   d2_anova %>% filter(country == "US")); r2.5b_us

    Paired t-test

data:  prop_believeX by super_cat
t = 11.476, df = 70, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 0.3312579 0.4706201
sample estimates:
mean of the differences 
               0.400939 

# Ghana
r2.5b_gh <- t.test(prop_believeX ~ super_cat, paired = T,
                   d2_anova %>% filter(country == "Ghana")); r2.5b_gh

    Paired t-test

data:  prop_believeX by super_cat
t = 4.0562, df = 45, p-value = 0.0001957
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 0.05691244 0.16917452
sample estimates:
mean of the differences 
              0.1130435 

# Thailand
r2.5b_th <- t.test(prop_believeX ~ super_cat, paired = T,
                   d2_anova %>% filter(country == "Thailand")); r2.5b_th

    Paired t-test

data:  prop_believeX by super_cat
t = 9.7482, df = 97, p-value = 4.691e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 0.2210417 0.3340604
sample estimates:
mean of the differences 
               0.277551 

# China
r2.5b_ch <- t.test(prop_believeX ~ super_cat, paired = T,
                   d2_anova %>% filter(country == "China")); r2.5b_ch

    Paired t-test

data:  prop_believeX by super_cat
t = 12.57, df = 99, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 0.2880159 0.3959841
sample estimates:
mean of the differences 
                  0.342 

# Vanuatu
r2.5b_vt <- t.test(prop_believeX ~ super_cat, paired = T,
                   d2_anova %>% filter(country == "Vanuatu")); r2.5b_vt

    Paired t-test

data:  prop_believeX by super_cat
t = 6.8338, df = 72, p-value = 2.235e-09
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 0.1264582 0.2306194
sample estimates:
mean of the differences 
              0.1785388 

The difference between question categories was significant in each country/language considered alone.

Free response data

Here’s a very quick pass at looking at the most common words/phrases in these free responses – I did a quick and dirty “stemming” (converting, e.g., “believe” and “believes” and “believed” all to the stem “believ”) but we could look into doing something more sophisticated. Here are the top 5 stems for each question category, by country:

country	super_cat	category	response_lemma2	percent	n
US	religious	Christian religious	believe	52%	184
			think	21%	74
			say	10%	37
			know	5%	17
			read	2%	8
		Buddhist religious	believe	43%	154
			think	27%	95
			say	9%	32
			know	4%	15
			learn	4%	15
	fact	well-known fact	know	45%	158
			think	18%	65
			believe	10%	35
			say	8%	29
			learn	5%	16
		esoteric fact	think	26%	92
			know	19%	69
			say	11%	38
			believe	9%	33
			learn	8%	29
		life fact	think	28%	100
			say	16%	58
			know	15%	55
			forget	5%	16
			believe	4%	15
Ghana	religious	Christian religious	know	29%	67
			believe	17%	40
			say	16%	36
			hear	12%	27
			see	7%	17
		Buddhist religious	know	23%	52
			say	17%	40
			believe	12%	28
			see	12%	28
			think	12%	27
	fact	well-known fact	know	35%	81
			hear	15%	34
			say	13%	30
			think	10%	23
			see	6%	14
		esoteric fact	know	33%	77
			say	15%	35
			think	14%	33
			hear	12%	28
			see	6%	14
		life fact	know	23%	52
			say	18%	41
			think	14%	32
			see	11%	26
			not know	8%	18
Thailand	religious	Christian religious	believe	33%	164
			think	26%	126
			tell	12%	57
			know	6%	31
			say	4%	19
		Buddhist religious	believe	31%	153
			think	24%	117
			know	15%	72
			tell	11%	54
			say	4%	20
	fact	well-known fact	know	33%	160
			think	24%	117
			tell	17%	82
			believe	7%	34
			say	5%	24
		esoteric fact	think	32%	157
			know	19%	93
			tell	12%	60
			imagine	6%	29
			believe	5%	24
		life fact	think	36%	178
			tell	21%	101
			know	17%	85
			believe	4%	18
			expect	3%	14
China	religious	Christian religious	believe	40%	202
			think	29%	146
			feel	3%	17
			do not believe	3%	14
			[adverb] believe	2%	10
			say	2%	10
		Buddhist religious	think	31%	156
			believe	27%	135
			know	5%	23
			feel	3%	17
			do not believe	3%	14
	fact	well-known fact	know	57%	285
			think	11%	55
			believe	6%	29
			say	4%	22
			be sure of	3%	13
		esoteric fact	know	32%	160
			think	17%	86
			do not know	5%	27
			say	5%	23
			believe	4%	20
		life fact	know	18%	91
			discover	8%	42
			say	8%	41
			think	8%	41
			find	6%	32
			remember	6%	32
Vanuatu	religious	Christian religious	believe	31%	114
			say	19%	69
			know	8%	28
			think	7%	27
			pray	2%	9
		Buddhist religious	say	26%	94
			believe	16%	59
			see	11%	39
			think	9%	34
			know	5%	19
	fact	well-known fact	know	21%	76
			say	21%	76
			think	9%	34
			believe	8%	31
			find out	4%	13
		esoteric fact	say	22%	81
			think	12%	44
			know	11%	39
			believe	7%	26
			see	6%	21
		life fact	say	29%	105
			know	18%	64
			think	8%	29
			want	6%	23
			forget	5%	19

I think there’s lots to discuss here – e.g., the common use of “know” (which I think is already of interest). Also, the Bislama data appears to be in Bislama (not translated) – I’ve included “bilif” (and spelling variants) as “believe” and “ting” (as spelling variants) as “think” in all of the foregoing analyses.

Column `response_lemma2` joining factor and character vector, coercing into character vector

Analysis: Religion and religiosity

Demographics

First, let’s just look at how people in different countries replied to the relevant questions.

thb2_demo_regp: “Are you a part of any religious group?”

thb2_demo_rely: “From 1 to 7, how religious are you? (1 = not religious at all, 7 = extremely religious)”

Seems to have been omitted in Thailand?

thb2_demo_impr: “From 1 to 7, how important to you is your religious practice? (1 = not important at all, 7 = of utmost importance)”

Seems to have been omitted in Thailand?

thb2_demo_wors: “How often do you attend places of worship?”

thb2_demo_bgod: “What best describes your level of belief in God?”

thb2_demo_bbuh: “What best describes your level of belief in Buddha?”

thb2_demo_bosp: “What best describes your level of belief in another spiritual being (other than God or Buddha)?”

thb2_demo_atsn: "What best describes your attitude towards the supernatural?

thb2_demo_imsn: “From 1 to 7, how important to you is your attitude toward the supernatural? (1 = not important at all, 7 = of utmost importance)”

Analyses

Now, let’s look at how responses to our think/believe questions might have varied depending on religiosity/etc. For now, I’ll just focus on a couple of variables that seem to have been answered in reasonable ways.

thb2_demo_rely: “From 1 to 7, how religious are you? (1 = not religious at all, 7 = extremely religious)”

r2.6 <- lmer(believe ~ super_cat * country * thb2_demo_rely_num
             + (1 + super_cat | thb2_subj) + (1 | question),
             data = d2_long %>% 
               filter(country != "Thailand") %>%
               mutate(thb2_demo_rely_num = scale(thb2_demo_rely_num)),
             contrasts = list(country = "contr.sum"))

contrasts dropped from factor country due to missing levelscontrasts dropped from factor country due to missing levelscontrasts dropped from factor country due to missing levelscontrasts dropped from factor country due to missing levels

Parameter	β	β'	β''	Std. Err.	df	t	p
Intercept	0.17	-	-	0.02	101.49	10.03	<0.001	***
Category (religious)	0.12	0.33	0.33	0.02	70.54	7.98	<0.001	***
Country (US)	0.11	0.20	0.20	0.02	281.00	5.56	<0.001	***
Country (Ghana)	-0.11	-0.19	-0.19	0.03	281.00	-3.61	<0.001	***
Country (China)	0.01	0.03	0.03	0.02	281.00	0.68	0.498
How religious are you?	0.00	0.01	0.01	0.01	281.00	0.25	0.805
Category (religious) × Country (US)	0.07	0.14	0.14	0.02	281.00	4.78	<0.001	***
Category (religious) × Country (Ghana)	-0.07	-0.13	-0.13	0.02	281.00	-3.03	0.003	**
Category (religious) × Country (China)	0.03	0.06	0.06	0.02	281.00	1.58	0.116
Category (religious) × How religious are you?	0.00	-0.01	-0.01	0.01	281.00	-0.30	0.768
Country (US) × How religious are you?	0.00	0.01	0.01	0.02	281.00	0.20	0.841
Country (Ghana) × How religious are you?	0.02	0.04	0.04	0.02	281.00	0.92	0.359
Country (China) × How religious are you?	0.00	0.00	0.00	0.02	281.00	-0.11	0.914
Category (religious) × Country (US) × How religious are you?	0.00	-0.01	-0.01	0.02	281.00	-0.26	0.796
Category (religious) × Country (Ghana) × How religious are you?	0.01	0.02	0.02	0.02	281.00	0.50	0.618
Category (religious) × Country (China) × How religious are you?	0.00	0.01	0.01	0.02	281.00	0.22	0.823

This analysis suggests that greater religiosity was NOT associated with an increased distinction between religious and fact questions. (Note that this analysis omits participants from Thailand, who did not answer this question about religiosity.)

thb2_demo_impr: “From 1 to 7, how important to you is your religious practice? (1 = not important at all, 7 = of utmost importance)”

r2.7 <- lmer(believe ~ super_cat * country * thb2_demo_impr_num
             + (1 + super_cat | thb2_subj) + (1 | question),
             data = d2_long %>% 
               filter(country != "Thailand") %>%
               mutate(thb2_demo_impr_num = scale(thb2_demo_impr_num)),
             contrasts = list(country = "contr.sum"))

contrasts dropped from factor country due to missing levelscontrasts dropped from factor country due to missing levelscontrasts dropped from factor country due to missing levelscontrasts dropped from factor country due to missing levels

Parameter	β	β'	β''	Std. Err.	df	t	p
Intercept	0.17	-	-	0.02	94.36	10.18	<0.001	***
Category (religious)	0.12	0.32	0.32	0.02	65.40	7.77	<0.001	***
Country (US)	0.10	0.20	0.20	0.02	280.03	5.43	<0.001	***
Country (Ghana)	-0.09	-0.16	-0.16	0.03	280.03	-3.27	0.001	**
Country (China)	0.02	0.04	0.04	0.02	280.03	0.89	0.372
How important is your religious practice?	0.00	0.00	0.00	0.01	280.03	0.14	0.887
Category (religious) × Country (US)	0.08	0.15	0.15	0.02	280.03	4.97	<0.001	***
Category (religious) × Country (Ghana)	-0.07	-0.13	-0.13	0.02	280.03	-3.10	0.002	**
Category (religious) × Country (China)	0.03	0.07	0.07	0.02	280.03	2.02	0.045	*
Category (religious) × How important is your religious practice?	0.00	0.00	0.00	0.01	280.03	0.01	0.989
Country (US) × How important is your religious practice?	-0.02	-0.03	-0.03	0.02	280.03	-0.88	0.382
Country (Ghana) × How important is your religious practice?	0.01	0.02	0.02	0.02	280.03	0.52	0.603
Country (China) × How important is your religious practice?	0.00	0.00	0.00	0.02	280.03	0.02	0.983
Category (religious) × Country (US) × How important is your religious practice?	-0.02	-0.04	-0.04	0.02	280.03	-1.55	0.123
Category (religious) × Country (Ghana) × How important is your religious practice?	0.01	0.02	0.02	0.02	280.03	0.61	0.545
Category (religious) × Country (China) × How important is your religious practice?	0.00	0.00	0.00	0.02	280.03	-0.08	0.934

This analysis suggests that more importance placed on religious practice was NOT associated with an increased distinction between religious and fact questions. (Note that this analysis omits participants from Thailand, who did not answer this question about religiosity.)

thb2_demowors: “How often do you attend places of worship?”

r2.8 <- lmer(believe ~ super_cat * country * thb2_demo_wors_num
             + (1 + super_cat | thb2_subj) + (1 | question),
             data = d2_long %>% 
               mutate(thb2_demo_wors_num = scale(thb2_demo_wors_num)))

Parameter	β	β'	β''	Std. Err.	df	t	p
Intercept	0.17	-	-	0.02	189.99	8.63	<0.001	***
Category (religious)	0.12	0.32	0.32	0.02	121.80	7.14	<0.001	***
Country (Gh.)	-0.12	-0.18	-0.18	0.04	364.98	-2.74	0.006	**
Country (Th.)	0.02	0.03	0.03	0.02	364.98	0.71	0.475
Country (Ch.)	0.02	0.04	0.04	0.04	364.98	0.55	0.586
Country (Vt.)	-0.03	-0.04	-0.04	0.04	364.98	-0.71	0.480
How often do you attend places of worship?	0.02	0.05	0.05	0.02	364.98	1.10	0.273
Category (religious) × Country (Gh.)	-0.08	-0.13	-0.13	0.03	364.99	-2.39	0.017	*
Category (religious) × Country (Th.)	0.01	0.02	0.02	0.02	364.99	0.79	0.431
Category (religious) × Country (Ch.)	0.02	0.04	0.04	0.03	364.99	0.80	0.425
Category (religious) × Country (Vt.)	-0.03	-0.05	-0.05	0.03	364.99	-0.96	0.336
Category (religious) × How often do you attend places of worship?	0.00	0.01	0.01	0.01	364.99	0.21	0.833
Country (Gh.) × How often do you attend places of worship?	0.02	0.04	0.04	0.04	364.98	0.61	0.544
Country (Th.) × How often do you attend places of worship?	0.02	0.02	0.02	0.03	364.98	0.44	0.657
Country (Ch.) × How often do you attend places of worship?	-0.01	-0.02	-0.02	0.03	364.98	-0.39	0.695
Country (Vt.) × How often do you attend places of worship?	-0.01	-0.02	-0.02	0.03	364.98	-0.33	0.739
Category (religious) × Country (Gh.) × How often do you attend places of worship?	0.02	0.03	0.03	0.03	364.99	0.54	0.587
Category (religious) × Country (Th.) × How often do you attend places of worship?	0.02	0.02	0.02	0.03	364.99	0.65	0.515
Category (religious) × Country (Ch.) × How often do you attend places of worship?	-0.01	-0.01	-0.01	0.02	364.99	-0.26	0.796
Category (religious) × Country (Vt.) × How often do you attend places of worship?	-0.01	-0.01	-0.01	0.03	364.99	-0.22	0.826

This analysis suggests that frequency of attendence was NOT associated with an increased distinction between religious and fact questions.